1 The PIPE: queue-handler

In the AmigaOS release 2.04, a new handler for passing data between programs was introduced. This handler is called ‘L:Queue-Handler’, better known as the ‘PIPE:’ device.

‘PIPE:’ implements a true pipe familiar from UNIX systems. Its function is to pass the output of one program to another program as input. You can chain several programs together using multiple pipes.

The advantages of using pipes are:

1. No need for a lot of RAM to keep the temporary files, or, no need to slow things down by storing temporary files on the HD.
2. Faster operation since the intermediate data doesn’t need to be stored anywhere between programs.

The ‘PIPE:’ device differs from the UNIX pipes in two important respects:

1. It is a device, so the input and output of the programs doesn’t need to be a redirected standard output, although that is the general case. You can also use ‘PIPE:’ instead of files, with two restrictions; you can not read a directory from ‘PIPE:’, and the ‘PIPE:’ ‘file’ can not be ‘seeked’ in.
2. It does not support a flush operation. If all of the data written in a ‘PIPE:’ device is not read out, it will stay there, buffered. Thus you must always ‘empty’ a pipe before closing it.
3. Because of the above, if unread data exceeds the size of the ‘PIPE:’ internal buffer, the program writing to the pipe will ‘block’ (that is, suspend operation) until the data is read. See below on how to manually flush a pipe.

The PIPE: handler name syntax is ‘PIPE:name/bufsize/bufnum’, where ‘name’ is the name for the pipe channel. Using names you can have several simultaneous pipe operations. The optional arguments bufsizeand bufnum specify the size and number of the buffers used by ‘PIPE:’. Normally, you would just use ‘PIPE:name’.

The default buffer size of ‘PIPE:’ is 4096 bytes, and the number of buffers (ie. channels) is unlimited.

Osma Ahvenlampi (Osma.Ahvenlampi@hut.fi)

1.1 Using PIPE: in a standard AmigaShell environment

First, make sure ‘PIPE:’ is mounted. You can do this by giving the command

    1> Mount PIPE:

in a Shell window. On an AmigaOS 2.1+ system, ‘PIPE:’ is normally mounted on startup by placing the mount entry file ‘PIPE’ in the drawer ‘DEVS:DosDrivers’.

In an AmigaShell window, give the commands:

    1> Run List SYS: >PIPE:Listoutput
    1> More <PIPE:Listoutput

(1)

These two commands list the contents of the ‘SYS:’ volume to a pipe and then type it in the Shell window. You could also use:

    1> Run List SYS: NOHEAD >PIPE:Listoutput
    1> Run Sort PIPE:Listoutput PIPE:Sortedoutput
    1> More <PIPE:Sortedoutput

Notice that you need to ‘Run’ all the programs except for the last one. This is so that you can get them all running simultaneously. You could also use several Shell windows, or any other way of running all the programs at the same time. Also this might not work if ‘More’ isn’t the standard Commodore AmigaOS 2.04+ More found in ‘SYS:Utilities’.

In case you need to manually flush a pipe, for example because the program that was reading from it aborts prematurely, it can be done with the command:

    1> Type PIPE:name TO NIL:

1.2 The Pipe command

As you can see, this is a somewhat difficult way of using pipes, needing a lot of typing especially compared to the UNIX pipes. There is a solution for that.

Andy Finkel, a former Commodore software engineer wrote a command called Pipe (not to be confused with ‘PIPE:’) that simplifies the use of the ‘PIPE:’ device. Unfortunately, this command did not make it into the 2.04 release. However, Commodore has given the permission to distribute this, and a few other commands, freely. (Sources: Fish disk 673, Aminet, ‘util/cli/finkelshelltools.lha’)

For some reason Commodore did not include these commands in the later OS releases, although they work fine even with OS 3.1 (V40). Thus all users are required to get them from the above sources if they wish to use them.

The Pipe command usage is simple. The commands to be run with their inputs and outputs piped together are given as arguments to the command, separated by the <|> (vertical bar) character:

    1> Pipe List SYS: | More
        or
    1> Pipe List: SYS: NOHEAD | Sort IN: OUT: | More

Notice how Sort needs ‘IN:’ and ‘OUT:’. These devices don’t actually exist, but the Pipe command simulates them with the PIPE:. This is necessary because C:Sort can not use the ‘standard IO’, but insists on files.

The separator character <|> can be configured to be something else with the local variable __pchar.

1.3 Pipe command support in AmigaShell

There is a little known and quite useful undocumented feature built in the 2.04+ AmigaShell, that is, AmigaShell knows about the Pipe command. By setting the local variable _pchar, you can use pipes without even entering the Pipe command. AmigaShell will prepend it on the command line automatically if it sees the pipe separator command specified by _pchar on the command line.

Thus, after the command:

    1> Set _pchar "|"

(2)

the above examples can be used like this:

    1> List SYS: | More
        or
    1> List SYS: NOHEAD | Sort IN: OUT: | More

Those familiar with the UNIX pipes will notice how the usage is very similar to the UNIX pipe usage.

1.4 Quick usage guide

1. Get the ShellTools archive from the Fish disk 673, or from Aminet, ‘util/cli/hacks204.lha’.
2. Install the commands in this archive in your Shell search path, preferably C:.
3. Put the command ‘Set _pchar |’ in your ‘S:Shell-Startup’ file.
4. Restart your AmigaShell so that it reads the ‘S:Shell-Startup’.
5. Try ‘List SYS: | More’
6. Read the ShellTools documentation and experiment.

1.5 Related things

The variable _mchar is used to separate one command from another in a shell line. So, after the command:

    1> Set _mchar ";"

you can put more shell commands into the same line, separated from each other by ;.

1.6 Troubleshooting

Q:

When I try the command ‘List SYS: | More’ I get an error

    PIPE: Unknown command

I have ‘PIPE:’ mounted, what’s wrong?

A:

AmigaShell is referring to the command Pipe in the error, not the ‘PIPE:’ device. You haven’t installed the commands in the ShellTools archive.

Q:

I get a requester that says

    Please insert volume PIPE: in any drive

when I try any of the command examples.

A:

‘PIPE:’ is not mounted. Try ‘Mount PIPE:’. If that doesn’t work, refer to your AmigaDOS manual or a friend that knows how to mount devices.

Q:

When I try any of the examples the More window pops up but nothing is shown/I get an error afterwards/More pops up a file requester.

A:

You are not using the standard AmigaOS More command. Either use another name or install a command that supports pipes. Possibilities are the C= More, Less, or Most for example.

Q:

With use of PIPE:, sometimes characters get lost. What happened?

A:

Up to OS 3.1, the queue-handler has some bugs which don’t allow a really foolproof transmission of data. A compatible replacement for the queue-handler can be founbd on Aminet, ‘util/sys/HWGQueue.lha’.

Q:

None of these happen but it still doesn’t work as you described.

A:

Are you using OS 2.04 later? Are you using the standard AmigaShell? Try with all extras disabled.

Footnotes

(1)

Note that ‘1>’ is the prompt AmigaShell gives when it is waiting for a command. The number varies, and the prompt can be customised to look completely different. It is not a part of the command.

(2)

The quotes around the vertical bar are important, to avoid AmigaShell interpreting the command as an attempt to use PIPE:, if _pchar is already set (for example if the Shell is started from within another one).

About This Document

This document was generated on February 12, 2023 using texi2html 5.0.

The buttons in the navigation panels have the following meaning:

where the Example assumes that the current position is at Subsubsection One-Two-Three of a document of the following structure:

• 1. Section One
  • 1.1 Subsection One-One
    • ...
  • 1.2 Subsection One-Two
    • 1.2.1 Subsubsection One-Two-One
    • 1.2.2 Subsubsection One-Two-Two
    • 1.2.3 Subsubsection One-Two-Three     <== Current Position
    • 1.2.4 Subsubsection One-Two-Four
  • 1.3 Subsection One-Three
    • ...
  • 1.4 Subsection One-Four

This document was generated on February 12, 2023 using texi2html 5.0.